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Tech Stocks: Follow the Patents? 



That's what one research boutique is doing-and its work suggests that investors should pay close attention to companies* patent activity 

Hungry for technology stocks? If that question leaves you feeling as if you just swallowed seafood of dubious vintage, I 
don ? t blame you. By any measure, the past two years have been enough to make even the stoutest investor shudder at tech. 
Yet abandoning technology companies, which account for one in five U.S. stocks, would be an obvious, if 
understandable, mistake. 

That's why I was all ears when I heard recently about a novel approach to picking techs. It has nothing to do with counting 
eyeballs, checking book-to-bill ratios, or even estimating earnings per share. Instead, it's all about patents-first, trying to 
evaluate the potential of a company's patent portfolio, and then relating that measurement to the company's stock price. 
Early evidence of patents' role in picking winners is promising (chart). 

Yes, many investors have long hazarded guesses about how key patents might pay off. But CHI Research, a consulting 
boutique in Haddon Heights, N.J., is pioneering a strictly quantitative method. CHI got started in 1968 reviewing patents 
for the National Science Foundation. It later nabbed corporate clients and, in 2000, branched out to Wall Street with a 
pricey stock-evaluation service it calls Investor Tech-Line. CHI each month examines nearly 400 companies with strong 
patent activity, ranking them on investment potential. 

Far more important than just the number of patents a company holds, or even how many it is adding, are a battery of more 
subtle indicators. At the core of CHI's system is something it calls "citation impact." This is a measure of how often a 
patent is cited by later patent applications and, by extension, its importance. At the same time, CHI checks how many 
references a patent makes to scientific papers, as opposed to earlier patents. Its hunch is that patents flowing out of 
academic work are closer to basic science and likely more valuable. The firm also measures what it calls "technology 
cycle time," or the median age of patents cited in an application. Citing more recent patents can indicate a company is 
dealing in fresher technologies, with the possibility of greater future profit. 

Tests of these theories, first reported in a 1999 paper published in the Financial Analysts Journal and later by CHI itself, 
are intriguing. CHI's back-test over the 1990-99 period found its picks outperformed the Nasdaq Composite Index in 7 of 
10 years and the Standard & Poor's 500-stock index in 8 of 10. 

Investor Tech-Line, for which CHI last year received its own patent (you can view it at www.chiresearch.com) , is slowly 
drawing the attention of institutional investors. Christopher Rowane, manager of the $60 million Huntington Mid Corp 
America Fund, attributes much of the fund's outperformance since its start a year ago— a 4.5% gain, vs. a 2.6% loss for its 
average midcap rival— to relying on patent research for a first cut through the clutter of stocks. 

So, which stocks does CHI like? Some surprises, notably Ciena (table). The maker of optical-network gear topped CHI's 
yearend rankings, when it was $14.31 a share, and again on Feb. 1, at $12.70. Ciena ( CIEN ) has since announced a 
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merger with ONI Systems ( QMS ) and revealed a bleak outlook for the current quarter, sending the shares below $8. 
Other CHI picks include well-known giants (AOL Time Warner) and tiny unknowns such as Isco ( ISKO ), a maker of 
flow meters. CHI suggests Isco's miniature, $54 million market value belies the close link of its patents to the science of 
"supercritical fluids"-part-liquid, part-gas substances created under very high pressure--that are used increasingly in the 
chemical and food industries. 

An annual subscription to CHI's monthly reports carries a $15,000 price tag. But its way of thinking about patents, future 
profits, and tech stocks is free-a new tool to add to your kit. 

By Robert Barker 



Copyright 2000-2004, by The McGraw-Hill Companies Inc. All rights reserved. 
Terms of Use P rivacy Notice 
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A Search Engine For Tech Prospectors 

It seems so far away. In March, 2002, tech investors were deep in a second straight winter of distress. The Nasdaq had 
been cut by two-thirds in just two years. Tech was positively cryogenic - and an irresistible draw for anyone hungry for 
companies with fresh technology. But which companies? 

A consulting firm called CHI Research makes finding those companies its business. Two years back, when I asked CHI to 
share its top 10 tech-stock picks with BusinessWeek readers, half the names on the list were new to me. Yet CHI was on to 
something. Since then, its picks have returned an average of 59.2%, while the Nasdaq 100 and Standard & Poor's 500 
indexes returned 4.2% and 6.1%, respectively. Each year since 2000, when CHI began issuing monthly buy 
recommendations to institutional investors (subscription price: $15,000 a year), it has killed the market averages. In 2003, 
as the average tech-stock mutual fund returned 55.9%, CHI's picks returned 162%. 



HOW DO THEY DO IT? CHI uses a strictly quantitative method based on evaluating the strength of public companies 1 
patent portfolios. Based in Haddon Heights, N.J., the firm got its start in 1968 reviewing patents for the National Science 
Foundation. It still consults for corporate clients, but its investment-research method, on which it has its own patent, is 
gaining renown. Here's how it works: Every month, CHI looks over the patents of 477 innovators, from mighty General 
Electric ( GE ) to the likes of genomics outfit Lynx Therapeutics ( LYNX ). It checks not just how many patents each 
company holds but what it refers to as "citation impact" - how often they are cited in later patent applications. CHI also 
checks how many references a patent makes to academic papers, a way to see how closely to basic science the company is 
working. And it notes how quickly the companies exploit scientific advances by checking the median age of patents cited 
in their patent applications. Finally, CHI considers the companies' stock-price-to-book-value ratios. 

Naturally, CHI's research isn't foolproof. Of the 10 picks listed in BusinessWeek in 2002, three had negative returns, 
including CHI's then- favorite, optical networker Ciena (CIEN ). Now, its favorites (table) include several biotech names, 
such as Maxygen ( MAXY ), Transgenomic ( TBIO ), and Caliper Life Sciences ( CALP ), a repeat from 2002. Patrick 
Thomas, a CHI senior analyst, said basic materials companies lately are climbing the ranks. One is Arch Chemicals 
( ARJ ), which Olin ( OLN ) spun off in 1999. CHI is keen on its patents for polishing copper-coated wafers and on the 
relatively low age of its patents — indicating fast innovation. 

Cabot, ( CBT ) another name on the list that many investors may not associate with high tech, is a longtime producer of 
carbon black, a key material used by tiremakers. But Cabot also sells specialty powders for ink jets, fuel cells, and flat- 
panel displays. And it knows how to cash in on innovation. In 2000 it spun off Cabot Microelectronics ( CCMP ), a 
supplier of polishing slurries and pads for microelectronics. Before the spin-off, Cabot had a market value of $2.2 billion. 
Within a year, the parent and offspring's total value topped $5 billion. 

Expecting Cabot to hit another such lode is unreasonable, just as it's unlikely the NASDAQ will have another year like 
2003 soon. Still, CHI's list strikes me as a good place for tech investors to keep prospecting. 



By Robert Barker 



http://www. businessweek. com:/print/magazine/content/04_10/b3873122_mz026. htm?mz 



3/5/04 



Record List Display 



http://westbrs:8002ftir^^ 



WEST 



Generate Collection 



Print 



Search Results - Record(s) 1 through 6 of 6 returned. 



□ 1. Document ED: US 20020084901 Al 

L4: Entry 1 of 6 File: PGPB 



Jul 4, 2002 



PGPUB -DOCUMENT -NUMBER : 20020084901 
PGPUB- FILING -TYPE : new 

DOCUMENT- IDENTIFIER: US 20020084901 Al 

TITLE: System and method for smart clothing and wearable electronic devices 
PUBLICATION-DATE: July 4, 2002 



INVENTOR- INFORMATION : 
NAME 

Manty j arvi , Jani 
Hoisko, Jyrki 
Kaario, Juha 
Kimmel, Jyrki 



CITY 

Espoo 

Oulu 

Tampere 

Tampere 



STATE 



US - CL - CURRENT : 340/573 . 1; 340/825 .36 




Classification 



COUNTRY 

FI 

FI 

FI 

FI 



RULE -4 7 



□ 2. Document ID: US 6466232 Bl 

L4: Entry 2 of 6 File: USPT 



Oct 15, 2002 



US-PAT-NO: 6466232 

DOCUMENT- IDENTIFIER: US 6466232 Bl 

TITLE: Method and system for controlling presentation of information to a user based 
on the user's condition 

DATE-ISSUED: October 15, 2002 
INVENTOR- INFORMATION : 

NAME CITY STATE ZIP CODE COUNTRY 

Newell; Dan 

Abbott, III; Kenneth H. 



CITY STATE ZIP CODE 

Seattle WA 
Kirkland WA 



US -CL- CURRENT: 345/700; 345/3.1, 345/751, 345/8, 345/866 



Record List Display 



http://westbrs:8002/b^^ 



□ 3. Document ID: US 6305221 Bl 

L4: Entry 3 of 6 



File: USPT 



Oct 23, 2001 



US-PAT-NO: 6305221 

DOCUMENT- IDENTIFIER: US 6305221 Bl 
TITLE: Rotational sensor system 
DATE-ISSUED: October 23, 2001 



INVENTOR- INFORMATION : 
NAME 

Hutchings; Lawrence J. 
US -CL- CURRENT: 73/488; 73/510 



CITY 

Castro Valley 



STATE ZIP CODE 
CA 



COUNTRY 




Full I Title | Citation | Front | Review | Class if ication 
Or aim Desc | Image 1^H|^^^^^^^^^^^^^^^^^^| 



□ 4. Document ID: US 6122960 A 

L4 : Entry 4 of 6 



File: USPT 



Sep 26, 2000 



US-PAT-NO: 6122960 

DOCUMENT- IDENTIFIER: US 6122960 A 

TITLE: System and method for measuring movement of objects 
DATE -ISSUED: September 26, 2 000 



INVENTOR- INFORMATION : 
NAME 

Hutchings; Lawrence J. 
Gross; Richard 
Jarpe; Stephen 



CITY STATE 

Castro Valley CA 

San Francisco CA 

Prescott Valley AZ 



ZIP CODE 



COUNTRY 



US -CL- CURRENT: 73/493; 73/510, 73/865 .4 



Sequences | Attachments | Claims] 



El 5. Document ID: US 5899963 A 

L4: Entry 5 of 6 File: USPT May 4, 1999 

US-PAT-NO: 5899963 

DOCUMENT- IDENTIFIER: US 5899963 A 

TITLE: System and method for measuring movement of objects 
DATE-ISSUED: May 4, 1999 

INVENTOR- INFORMATION : 



Record List Display 



http://westbrs:8002ftin/ga^^ 



NAME 

Hutchings; Lawrence J. 



CITY 

Castro Valley 



STATE ZIP CODE COUNTRY 
CA 



US -CL- CURRENT: 702/145; 700/14, 702/141, 702/142, 702/146, 702/149 



Review | Classification | Date | Reference | Sequences | Attach merits 



Draw Deso | Image | 



□ 6. Document ID: US 20020084901 Al 

L4: Entry 6 of 6 File: DWPI 



Jul 4, 2002 



DERWENT-ACC-NO: 2002-635608 
DERWENT - WEEK : 2 002 68 

COPYRIGHT 2 003 DERWENT INFORMATION LTD 

TITLE: Wearable electronic device and smart clothing activation control method 
involves integrating electronic circuits into zipper or hook on clothing based on 
whose position electronic device is controlled 

Basic Abstract Text (9) : 

DESCRIPTION OF DRAWING (S) - The figure shows the block diagram of the wearable 
device and the smart clothing activation control system. 

Standard Title Terms (1) : 

WEAR ELECTRONIC DEVICE SMART CLOTHING ACTIVATE CONTROL METHOD INTEGRATE ELECTRONIC 
CIRCUIT HOOK CLOTHING BASED POSITION ELECTRONIC DEVICE CONTROL 



Full | Title I Citation | Front | Review 



Draw. Desc Image 



lassrfieation | Date | Reference | Sequenc 



Attachments 



Generate Collection 



Print 



Term 


Documents 


SMART.DWPI,TDBD,EPAB,JPAB,USPT,PGPB. 


32470 


SMARTS.DWPI,TDBD,EPAB,JPAB,USPT,PGPB. 


137 


CLOTHING.DWPI,TDBD,EPAB,JPAB,USPT,PGPB. 


69851 


CLOTHINGS.DWPI,TDBD,EPAB,JPAB,USPT,PGPB. 


977 


(SMART ADJ 

CLOTHING).USPT,PGPB,JPAB,EPAB,DWPI,TDBD. 


6 


(SMART CLOTHING).USPT,PGPB,JPAB,EPAB,DWPI,TDBD. 


6 



Display Format : 1 - 

Previous Page 



Change Format | 
Next Page 



Record List Display 




http://westbrs:8002ftin/gat^ 



MUST 



Generate Collection 



Print 



Search Results - Record(s) 1 through 9 of 9 returned. 



0 1. Document ID: US 6341372 Bl 

L10: Entry 1 of 9 File: USPT 



Jan 22, 2002 



US-PAT-NO: 6341372 

DOCUMENT- IDENTIFIER: US 6341372 Bl 

TITLE: Universal machine translator of arbitrary languages 
DATE -ISSUED: January 22, 2 002 



INVENTOR- INFORMATION : 
NAME 

Datig; William E. 



CITY 

Centerport 



STATE ZIP CODE 
NY 11721 



US -CL- CURRENT: 717/136; 715/523 



COUNTRY 




□ 2. Document ID: US 6233545 Bl 

L10: Entry 2 of 9 File: USPT May 15, 2001 

US-PAT-NO: 6233545 

DOCUMENT- IDENTIFIER: US 6233545 Bl 

TITLE: Universal machine translator of arbitrary languages utilizing epistemic 
moments 

DATE -ISSUED: May 15, 2 001 
INVENTOR- INFORMATION : 

NAME CITY STATE ZIP CODE COUNTRY 

Datig; William E. Centerport NY 11721 

US -CL- CURRENT: 704/2; 704/9, 706/62 



KViilC 



Full [ Title I Citation j Front j Review | Classification | Date [ Reference | Sequences | Attachments 
L>ra^rDe7<r7n^ 



□ 3. Document ID: US 5722418 A 

L10: Entry 3 of 9 File: USPT 



Mar 3, 1998 



Record List Display http://westbrs:8002Mn/g^ 




1 



US-PAT-NO: 5722418 

DOCUMENT- IDENTIFIER: US 5722418 A 



TITLE: Method for mediating social and behavioral processes in medicine and business 
through an interactive telecommunications guidance system 

DATE- ISSUED: March 3, 1998 
INVENTOR- INFORMATION : 

NAME CITY STATE ZIP CODE COUNTRY 

Bro; L. William Los Angeles CA 90045 

US -CL- CURRENT: 600/545; 128/905, 128/920, 434/118, 482/9, 600/300, 600/544 



Full | Title | Citation | Front | Review | Classification | Date | Reference | Sequences | Attachments 
Draw. Desc | lmage']fl|^^^^^^^^^^^^^^^^^^^^^HB||||^^^^^H^^^^^^^|^^^^^^^^^^^^^| 



□ 4. Document ID: US 5629855 A 

L10: Entry 4 of 9 



File: USPT 



May 13, 1997 



US-PAT-NO: 5629855 

DOCUMENT- IDENTIFIER: US 5629855 A 

TITLE: System and method for using parabolic models to improve position estimates 
from a global positioning system 

DATE- ISSUED: May 13, 1997 



INVENTOR- INFORMATION : 
NAME 

Kyrtsos; Christos T. 
Sennott; James W. 
Gudat; Adam J. 
Christensen; Dana A. 
Friedrich; Douglas W. 
Stafford; Darrell E. 



CITY 


STATE 


Peoria 


IL 


Bloomington 


IL 


Edelstein 


IL 


Peoria 


IL 


Pekin 


IL 


Dun lap 


IL 



ZIP CODE 



COUNTRY 



US -CL- CURRENT: 701/300; 340/988, 342/357.02, 342/450, 701/120, 701/215 



sequences Attachments 



Full 



~| Title | Citation | Front | Review | Classification | Date 



Draw. Deso | Image || 



□ 5. Document ID: US 5555503 A 

L10: Entry 5 of 9 File: USPT Sep 10, 1996 

US-PAT-NO: 5555503 

DOCUMENT- IDENTIFIER: US 5555503 A 

TITLE: System and method for providing accurate vehicle positioning using spatial 
bias techniques 

DATE-ISSUED: September 10, 1996 



Record List Display 




http://westbrs:8002/bin/gate.exe?f=TOC&.. 



1 2&ref = 1 0&dbname=USPT&ESNAME= 



INVENTOR- INFORMATION : 
NAME 

Kyrtsos; Christos T. 
Gudat; Adam J. 
Christensen; Dana A. 
Friedrich; Douglas W. 
Stafford; Darrell E. 



CITY 


STATE 


Peoria 


IL 


Edelstein 


IL 


Peoria 


IL 


Pekin 


IL 


Dun lap 


IL 



ZIP CODE 



COUNTRY 



US -CL- CURRENT: 701/207; 342/357.08, 342/457, 701/300 



Classification 



Attachments 



□ 6. Document ED: US 5438517 A 

L10: Entry 6 of 9 File: USPT 

US-PAT-NO: 5438517 

DOCUMENT- IDENTIFIER: US 543 8517 A 

TITLE: Vehicle position determination system and method 
DATE- ISSUED: August 1, 1995 

INVENTOR- INFORMATION : 

NAME CITY STATE ZIP CODE 

Sennott; James W. 
Kyrtsos; Christos T. 
Gudat; Adam J. 
Christensen; Dana A. 
Friedrich; Douglas W. 
Stafford; Darrell E. 

US -CL- CURRENT: 701/213; 342 / 357.03 , 342/457, 701/214, 701/226 



Aug 1, 1995 



CITY 


STATE 


Bloomington 


IL 


Peoria 


IL 


Edelstein 


IL 


Peoria 


IL 


Pekin 


IL 


Dun lap 


IL 



COUNTRY 



Full J Title | Citation | Front | Review ) Classification | Date j Reference | Sequences | Attachments 



□ 7. Document ID: US 5390125 A 

L10: Entry 7 of 9 File: USPT Feb 14, 1995 

US-PAT-NO: 5390125 

DOCUMENT- IDENTIFIER: US 5390125 A 

TITLE: Vehicle position determination system and method 
DATE -ISSUED: February 14, 1995 

INVENTOR- INFORMATION : 



Record List Display 




NAME 


CITY 


STATE 


Sennott; James W. 


Bloomington 


IL 


Kyrtsos; Christos T. 


Peoria 


IL 


Gudat; Adam J. 


Edelstein 


IL 


Chris tensen; Dana A. 


Peoria 


IL 


Friedrich; Douglas W. 


Pekin 


IL 


Stafford; Darrell E. 


Dun lap 


IL 



htrp://westbrs: 8002ftin/gate.exe?f==TOC&. . . i4jgp. 1 2&ref = 1 0&dbname=USPT&ESNAME= 



ZIP CODE 



COUNTRY 



US -CL- CURRENT: 701/214; 342/357 .02, 342/457, 701/226 



Full | Title I Citation | Front | Review | Classification | Date 



□ 8. Document ID: US 5375059 A 

L10: Entry 8 of 9 File: USPT Dec 20, 1994 

US-PAT-NO: 5375059 

DOCUMENT- IDENTIFIER: US 5375059 A 

TITLE: Vehicle position determination system and method 



ZIP CODE COUNTRY 



DATE -ISSUED: December 20, 1994 






INVENTOR- INFORMATION : 






NAME 


CITY 


STATE 


Kyrtsos; Christos T. 


Peoria 


IL 


Gudat; Adam J. 


Edelstein 


IL 


Chris tensen; Dana A. 


Peoria 


IL 


Friedrich; Douglas W. 


Pekin 


IL 


Stafford; Darrell E. 


Dunlap 


IL 


US -CL- CURRENT: 701/215; 342/357. 


.03, 342/457, 


701/226 



Full | Title | Citation | 


Front | 


R evieu'i 


} | Classification | 


Draw. Desc | image 









Date | Reference | sequences | Attachments 



□ 9. Document ID: US 4907170 A 

L10: Entry 9 of 9 File: USPT Mar 6, 1990 

US-PAT-NO: 4907170 

DOCUMENT- IDENTIFIER: US 4907170 A 

TITLE: Inference machine using adaptive polynomial networks 
DATE-ISSUED: March 6, 1990 



INVENTOR- INFORMATION : 
NAME 

Bhattacharya; Bishnu P. 



CITY STATE ZIP CODE COUNTRY 

Claremont CA 



Record List Display http://westbrs:8^^ 

US -CL- CURRENT: 706/14; 700/29, 700/82, 706/23, 706/31, 706 /44 



Full 


Title 


Citation 




Date | Reference 


Sequences 


Attachments | 



D 



Draw Desc Image 



Generate Collection 



| Print 



Term 


Documents 


ROBOTICS.USPT. 


5642 


ROBOTIC.USPT. 


12130 


(9 AND ROBOTICS).USPT. 


9 


(L9 AND ROBOTICS).USPT. 


9 



Display Format : | cit 

Previous Page 



] | Change Format 
Next Page 



http://portal.acniorg/resultsxfm?coll^ortal&dl=ACM&CFID=34723 1 8&CFTOKEN=3300 



: > home : > about : > feedback : > login 
US Patent & Trademark Office 



Search Results : 

Search Results for: ["smart clothing" and "wearable multimedia computing"] 
Found 3 of 105,778 searched. > Rerun within the Portal 

Search within Results 




> Advanced Search : > Search Help/Tips 



Sort by: Title Publication Publication Date Score Binder 



Results 1 - 3 of 3 short listing 



1 “Smart clothing”: wearable multimedia computing 82% 
2) and “personal imaging” to restore the technological 

balance between people and their environments 
Steve Mann 

Proceedings of the fourth ACM international conference on Multimedia 
February 1997 

2 Networked surfaces: a new concept in mobile networking 77% 
James Scott , Frank Hoffmann , Mike Addlesee , Glenford Mapp , Andy 
Hopper 

Mobile Networks and Applications October 2002 

Volume 7 Issue 5 

Networked Surfaces are surfaces which provide networking to 
specially augmented objects when these objects are physically 
placed on top of the surface. When an object (e.g., a notebook 
computer) connects, a handshaking protocol assigns functions 
such as data or power transmission to the various conducting 
paths that are established. This paper describes the position 
occupied by this concept in the world of networking, presents an 
overview of the technology used in its realisation, describes the ... 

3 Tangible bits: towards seamless interfaces between people, bits 77% 
12 and atoms 

Hiroshi Ishii , Brygg Ullmer 



0 PORTAL 



Results http://portaLacraorg/results.cfm?coll=portal&dl=ACM&CFID=34723 1 8&CFTOKEN=3300 

conference proceedings on Human factors in computing systems 
March 1997 



Results 1 - 3 of 3 short listing 

The ACM Portal is published by the Association for Computing Machinery. Copyright © 2003 
ACM, Inc. 



'Smart Clothing': Wearable Multimedia Computing and 
'Personal Imaging' to Restore the Technological Balance 
Between People and Their Environments * 

Steve Mann 
steve@media.irdt . edu 
MIT Media Lab, Building E15-389, 20 Ames Street, Cambridge, MA02139 

http://wearcam.org/ 



ABSTRACT 

Current portable computers and PDAs fail to truly become 
part of our daily lives in the sense that we need to stop what 
we are doing and expend conscious effort to use them. They 
also do not have the situational awareness that they should 
have: while they are not being explicitly used, they are unable 
to remain attentive to possible ways to help the user. 

Environmental technology in the form of ubiquitous com- 
puting, ubiquitous surveillance, and smart spaces, has at- 
tempted to bring multimedia computing seamlessly into our 
daily lives, promising a future world with cameras and mi- 
crophones everywhere, connected to invisible computing, al- 
ways attentive to our every movement or conversation. This 
raises some serious privacy issues. Even if we ignore these 
issues, there is still a problem of user-control, customization, 
and reliance on an infrastructure that will not (and probably 
should not) become totally ubiquitous. 

In response to these problems, a personal, wearable, multi- 
media computer, with head-mounted camera(s)/display, sen- 
sors, etc. is proposed for use in day-to-day living within 
the surrounding social fabric of the individual. Examples of 
practical uses include: face identification (memory aid for 
names), way-finding via sequences of freeze-frames, shared 
visual memory/environment maps, and other personal note- 
taking together with visual images. 

Anecdotal personal experiences, over several years of use, 
are reported, and privacy issues are addressed, in particu- 
lar, with a discussion of how personal 'smart clothing' has 
counteracted or at least reached a healthy balance with envi- 
ronmental surveillance. 

KEYWORDS: augmented reality, mediated reality, ubiqui- 
tous computing, smart spaces, video surveillance, mobile 
multimedia, wearable computing, personal imaging, video 
orbits, pencigraphic image compositing. 

•Research supported, in part, by HP labs, Palo Alto 
Words or phrases in single quotes are those introduced by author here or 
elsewhere tn the literature. 

Permission to make digital/hard copies of all or part of this material for 
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PROBLEM STATEMENT 

The advent of the personal computer brought computers 
closer to us — from distant computer rooms of the mainframe 
era right to our desks. However, computing still remains ac- 
cessible only when we're sitting at our desks, and is thus not 
really part of our personal day-to-day living. 

Later, portable computing made it possible to carry this 
personal environment in a briefcase or pocket. However, 
laptop computers and PDAs fail to become part of our daily 
lives in the sense that we need to stop what we are doing to 
use them. They are far from providing a seamless interaction 
in the context of day-to-day living. 

Other devices that we often carry, such as cellular tele- 
phones, pagers, wristwatches, personal sound systems, tape 
recorders, camcorders, and perhaps pocket calculators or 
measurement instruments such as pocket multimeters dupli- 
cate much of the same functionality many times over, and 
don't communicate with each other. These items are far more 
encumbering than would be a single item that performed all 
of the tasks that each was meant to do. 

Other developments, such as ubiquitous computing [1] have 
attempted to bring computing seamlessly into our daily lives. 
Ubiquitous multimedia computing and smart spaces [2] [3] [4] 
would seem to suggest a future world in which we're sur- 
rounded with computing, as well as cameras, microphones, 
and other forms of perceptual intelligence during all facets of 
our daily lives. 

There are two problems with smart spaces: (1 ) Not all envi- 
ronments will ever be so equipped. Even if most spaces were 
equipped with cameras and microphones, they may not serve 
our needs or be of direct benefit to us, as those who would 
install the infrastructure could not possibly predict our needs 
and exact preferences. Even if we desired that they know 
everything that could be known about us (and provided as 
much information as possible to them), it is still doubtful that 
their systems would totally eliminate the need for personal 
technology. (2) There is no guarantee that the organizations 
behind the infrastructure would not put their needs before 
ours. In particular, the prospect of ubiquitous surveillance 
would no doubt be attractive to many organizations, and the 
temptations to use it for other purposes (besides helping the 
users of the space) suggests that we might not want a world in 
which our every movement and conversation is being moni- 
tored by an external entity. 
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Morover, there are numerous well-documented examples in 
which organizations have abused their capability to covertly 
monitor their members. For example, it was recently reported 
that Dunkin Donuts was monitoring and recording conversa- 
tions within their establishments, using hidden microphones, 
in addition to their surveillance cameras, and the use of hid- 
den cameras in department store change rooms and employee 
locker rooms has also been documented[5][6]. 

Environmental perceptual intelligence is not new. Smart 
floors, electric eyes, motion sensors, smart lights, etc., have 
been used for a number of years. Some examples of smart 
spaces appear in Fig 1. However, as perceptual intelligence 
becomes networked in a multimedia environment, there will 
be more cameras installed and a wider variety of officially 
stated reasons for their existance. As cameras are installed 
for various purposes, people should be aware of the (very 
likely) possibility that they will have a plurality of purposes. 
Indeed, many video conferencing systems and multimedia 
computers have means to cover up cameras when they are 
not in use. This is because many people feel uncomfort- 
able with an exposed camera lens pointed at them, even if 
the camera is turned off. People are justified in this feeling; 
multimedia/visual intelligence systems should be designed 
with physical lens covers and not just on/off switches (in 
good designs the on/off switch and lens cover are combined 
so that sliding the cover over the lens automatically shuts 
off the power). The problem statement, in summary, is that 
neither laptop computers and PDAs, nor ubiquitous multime- 
dia computing/surveillance offer the kind of truly personal 
environment that would best suit our needs. 

PROPOSED SOLUTION: 'SMART CLOTHING' 

'Smart clothing*, the combination of wearable multimedia 
computing, personal imaging (through the use of one or more 
wearable video cameras), and wireless communications, into 
a rig that is comfortably worn in an active "always ready" 
mode, not just carried in a briefcase or the like — is pro- 
posed as an alternative to being forced to make a choice 
between portable multimedia computers/PDAs, and environ- 
mental technologies such as ubiquitous computing/surveillance. 

'Smart clothing* is a step toward truly personal computing 
and enhanced situational awareness, with less (or possibly 
no) reliance on a centralized infrastructure. 

The first 'smart clothing* prototype, designed and built by 
the author (Fig 2), comprised a modular personal, wearable, 
multimedia computer system together with one or more head- 
mounted cameras, a head-mounted display, and other sensors 
(one or more microphones, biosensors, two wearable radar 
systems, etc.), connected wirelessly to the Internet. The 
modular nature of the system allows portions to be left out or 
included, depending on the occasion. 

Currently, 'smart clothing* provides the following function- 
alies: 

• Time/date/calendar/reminder. replaces wristwatch but with addi- 
tional functionality such as meaningful alarm messages and online 
networked scheduling. 

• voice communications: replaces cellular phone or the like. 

• messaging: replaces pocket pager, but with much more informa- 
tive messaging (e.g. email). 




Figure 2: Wearable computer systems designed and built by author for 
experiments in personal imaging: Early (1980) apparatus was somewhat 
cumbersome. Before the advent of the TNCs in 1981, communications was 
handled with two separate radios at each end. The bulky 1.5 inch CRT 
required a bicycle helmet for support, and could only display 40 characters 
per row of text Later, a waist-mounted television was found to be less 
cumbersome, but failed to provide the constancy of user-interface. With the 
advent of miniature CRTs in the late 1980s, a comfortable eyeglass-based 
system became practical (1990), which was recently transferred to a more 
modern (early nineties) visor (VVSport TV, replacing the LCD with a CRT). 
A single hat-mounted antenna provided communications in the ham bands. 
More recently (1995) with the advent of cellular communications, a base- 
level of connectivity (at reduced speed) remains in effect when the unit is too 
far from its home base to use the high speed ham radio unit. Furthermore, in 
applications where the high speed ham radio link is not necessary, users of 
this technology will not need to pass a radio exam or obtain a special radio 
license as the commercially available cellular technology may be used. 

• sound capability: "Sound Blaster" compatible device replaces 
personal sound system. 

• sound capture: replaces dictating machine or the like. 

• video capture: replaces camcorder. Motion video capture capa- 
bility (e.g. AVI files, etc.) partially eliminates the need for a cam- 
corder. Striping/RAID on multiple SCSI disks (supported in the 
Linux operating system) around a belt or the like will soon facilitate 
digital video with full-motion, full-color, at maximum resolution. 

• pencigraphic image compositing: replaces 35mm still camera. 
By seamlessly "stitching together" multiple pictures of the same 
scene, the current system is capable of producing superresolution 
image composites in excess of 4000 pixels across. This resolution 
typically exceeds that attainable by a 35mm film camera. Since 
images may be transmitted wirelessly to friends and relatives (or 
others on the World Wide Web), when on vacation, there is no need 
to send traditional photo postcards or the like. 

• mathematical computation: replaces pocket calculator. Xcalc, 
be, Matlab, and Mathematica eliminate the need to carry a separate 
calculator or pocket computer. 

• measurement: replaces pocket volt, ohm, mi 11am meter, heart- 
rate monitor, etc. An 8 channel analog to digital converter provides 
voltage measurement (and measurement of other quantities with ap- 
propriate sensors, adapters etc.). This provides additional capability 
such as logging of heart rate (and full ECG waveform if desired), 
respiration, and the like. Quantities may be displayed numerically 
(like a digital voltmeter), in bargraph form (like a moving-coil volt- 
meter), or graphically (like an oscilloscope), eliminating the need to 
carry these 3 types of instruments. Automatic alert/program control 
of other functionality is also possible. 

Obviously these and other computational tools also provide 
not only far greater capability and integration of capability, 
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(a) 



(b) 



(c) 



(d) 



Figure 1 : Examples of "smart spaces", (a) "Intelligent highways" and ubiquitous surveillance. Systems like this are used for both traffic monitoring, and 
"public safety". In Baltimore, for example, the government is installing approximately 200 cameras throughout the city to keep watch over the general activities 
of its citizenry, (b) Smart ceilings (fifteen ceiling domes or dark windows visible here) monitor people in the space, purportedly for their benefit. Sophisticated 
machine vision algorithms are often used to track shoppers' activities [7] and make inferences about possible suspicious behaviour, (c) Machines with dark 
windows monitor users' activities, purportedly for the users' own protection, although organizations are often secretive about the exact nature of these systems 
(hence the use of very dark glass to hide the apparatus behind it), (d) 'Smart toilets', with dark windows, provide an awareness of the user's state to a miniature 
computer system or the like contained inside the box with the window, to assist the user in flushing the toilet. 



but they also provide totally new functionality. 
HISTORICAL BACKGROUND 

The proposed 'smart clothing' was not the world's first wear- 
able computer with wireless communications. Thomas Bass, 
in his book "The Eudaemonic Pie", describes shoe-based 
computers of the 1 970s that were designed and built by physi- 
cists and other researchers in California, for the purpose of 
assisting them at playing roulette. It was remarkable that 
they were able to design these computer systems to be so 
unobtrusive as to pass the ultimate test of unobtrusiveness - 
the "casino test" — surviving the scrutiny of the croupiers 
and pit bosses. It is also truly ironic that these computers 
were being used — privately — in a place where privacy 
would otherwise be unthinkable (under the 'eye in the sky', 
the partially-silvered ceiling mirrors and the ubiquitous ceil- 
ing domes of wine-dark opacity that are pervasive throughout 
most casinos). 

The author's suggestion of a community of nomads wearing 
wirelessly connected multimedia systems is not entirely new 
either. For many years, ham radio hobbyists have used voice 
communications on their hand-held battery-operated radios 
— often wearing them with headsets and hands-free boom 
microphones, sometimes even together with "antenna hats" 
quite similar to the author's — to stay in touch with each 
other as part of their day-to-day lifestyle, in all facets of their 
lives. Some even sent and received television pictures, using 
battery operated tetherless equipment. The ham radio com- 
munity, perhaps the predecessor of the internet newsgroups 
and the like, is also perhaps the predecessor of online wear- 
able wireless multimedia living. 

Ivan Sutherland, a pioneer in computer graphics, described 
a head-mounted display with half-silvered mirrors so that 
the wearer could see a virtual world superimposed on reality 
[8] [9]. Sutherland's work, as well as more recent related 
work [10][ll][12][13][14]is characterized by its tethered na- 
ture. The wearer is tethered to a workstation which is gen- 
erally powered from an AC outlet — in this sense it differs 



from the 'smart clothing* which is entirely battery operated 
and tetherless. 

OTHER RELATED WORK Other recent work in wearable 
computing[\5\ provides a task-specific system, in particular, 
a repair manual for use by soldiers. To make use simple, and 
to keep the soldier focused on the task at hand, the only input 
is a knob and pushbutton, so that menu items from a specific 
program may be selected. 

'Smart clothing' differs from employer-owned technology, 
or technology controlled by an external entity (the most ex- 
treme case being devices used to track criminals[16]). In 
particular, it is owned, operated, and controlled by the wearer. 

It is primarily intended for day-to-day living within the 
surrounding social fabric of the individual [17]. 

'PERSONAL IMAGING 1 THEORETICAL BACKGROUND 

The theoretical background for personal imaging is based 
on regarding the camera as a measurement instrument, in 
particular, an array of directional lightmeters. This theory is 
described in detail in[18][19][20]. Some of the basic ideas 
behind the theory are summarized here, in particular, those 
pertaining to automatic generation of image composites from 
multiple pictures of the same scene. 

PENCIGRAPHIC IMAGE COMPOSITING . Consider, for 
simplicity, a 1-D 'camera' in a 2-D static world. It is desired 
to know how two pictures, taken from the same location in 
space (but with different orientations, and possibly a change 
in focal length if the camera has zoom lens capability), are 
related to one another. The situation is depicted in Fig 3(a), 
where, for simplicity of visualization, the moving camera is 
depicted as two separate cameras drawn together on the same 
figure. From the figure, the coordinate transformation from 
x x to x 2 is given by 

X2 = Z2 tan(arctan(ari/2i) - $) y Vzi ^ o\ 

= (ax x +b)/{cx x +\) t Vx, (1) 
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Figure 3: Penrigraphic imaging; (a) TWo pictures of a static scene, taken from a common location, are related by a projectivity about the center of projection 
(COP). In particular, both images contribute to the same pencil of lines passing through the COP. The coordinate transformation from one image to the other is 
given by considering the mapping between domain x i and range xi . (b) This form of coordinate transformation gives rise to the 'projective chirping effect* [21], 
where uniformly spaced points in the domain (depicted as circles) map to a "chirping" lattice in the range (depicted by squares). In this example, the vanishing 
point (point of zero spatial frequency) is located at xa — 2, while the 'exploding point* (point of infinite spatial frequency) is given by the range of x \ = - 1 . 
(c) The plot of range coordinate as a function of the domain coordinate is in the form of a rectangular hyperbola, but shifted by the 'exploding point', o\ = — 1 
in the domain and the vanishing point, 02 = 2 in the range. A sinusoid is depicted in the domain together with the chirp function that it maps to in the 
range, (d) The 'projective chirping phenomenon' depicted on the face of a building having uniformly spaced windows. The best-fit chirp function captures this 
'periodicity-in-perspective*. 



where a — z%jz\ y b = ~Z2 tan(0), c = tan(#)/zi , and o\ = 
z\ tan(7r/2 + 6) = — 1/c, is the location of the singularity in 
the domain. 

Given a set of images that lie in the same orbit of the group, 
we wish to find for each image pair, that operator in the group 
which takes one image to the other image. 

If two frames, say, /1 and f 2f are in the same orbit, then 
there is an group operation p such that the mean-squared 
error (MSE) between f\ and f t — p o f 2 is zero. In practice, 
however, we find which element of the group takes one image 
"nearest" the other, for there will be a certain amount of 
parallax, noise, interpolation error, edge effects, changes in 
lighting, depth of focus, etc. 

A fully automatic featureless parameter estimator for esti- 
mating the parameters of the true projective group of coor- 
dinate transformations (e.g. the parameters in (1)) has been 
prpposed[ 18]. In particular: 
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where q2 = (be - a)c, qi = a - be - 1, and qo = 6, E x 
denotes the spatial derivative of the 'reference image', and 
E t the temporal derivative (between the two images). 

Images that arc in approximately the same orbit of the pro- 
jective group of coordinate transformations often arise from 
a quick movement of the head when wearing the apparatus 
described previously. An example of a *pencigraphic image 
composite* captured using WearCam, in which 1 17 images 
are seamlessly joined together into one larger image is shown 
in Fig 4. Other examples appear in 
ht tp : / /wearcam . org/pencigraphy /gallery . html 

The 'pencigraphic image representation' (regarding the cam- 
era as a collection of light-measuring instruments), together 
with the WearCam apparatus, forms the basis of 'personal 



imaging . 

HISTORICAL BACKGROUND Efforts to combine multiple 
images of the same scene have been around for many years. 
Photogrammeters have combined images manually, in non- 
overlapping sections 1 . Photogmmmetry is a well-developed 
field of study [22]. Artists, most notably, David Hockney [23], 
often assemble multiple pictures of the same scene by hand, 
using the medium expressively. 

Research in combining multiple pictures electronically has 
previously relied on an pure-translational model [24] of the 
form x 2 = x\ + b or an affine model[25] of the form xi = 
ax \ + 6. However, these models fail to capture the 'chirping* 
effect (described in the previous section) which is clearly quite 
pronounced in almost any practical imaging situation. With 
feature correspondences, the projective coordinate transfor- 
mation may be determined but it was not until 1993 that a 
fully automatic featureless method of doing so appeared in 
the literature[18]. 

Other related work, such as Apple's Quick Time VR (QTVR) 
(2) requires a special apparatus, comprising a tripod with a pre- 
cisely calibrated rotating camera stage, so that camera move- 
ments are known. Such an approach is unsuitable for the 
freeform 'painting with looks' approach to 'personal imag- 
ing'. 

EXPERIMENTAL APPARATUS 

Only a brief description of the apparatus is provided here; 
more detailed information can be found elsewhere in the 
literature[26]. 

The author's current apparatus, with miniature camera and 
miniature (one inch diag.) 24-bit color screen in the eye- 
glasses, and miniaturized Pentium 90 system (64M RAM, 



1 Images combined in non -overlapping sections (typically by mechanical 
means) are said to give rise to an image mosaic, whereas images that are com- 
bined together with overlap, feathering, or the like, typically by electronic 
means, are said to give rise to an image composite 
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Figure 4: A *pencigraphic image composite' created using the 'personal 
imaging* apparatus described in the previous section. As the author faced 
the cashier, and looked up toward the ceiling, 117 somewhat low-resolution 
NTSC images were captured. The processed image, showing the author's 
gaze pattern, is in some sense perhaps more like a painting than a photograph 
— the image contents are "painted" onto an empty image canvas merely 
by looking. Furthermore, even though the image was captured from a 
standard NTSC video camera, (giving 240-line images after deinterlacing), 
the resulting image is much larger (3730 pixels high) than any of the input 
images. Note the creative use of the distortionless (rectilinear) but extremely 
wide-angle perspective — the ceiling dome looming almost directly overhead 
is put near the 'exploding point' for maximum resolution enhancement 



1 .2G hard drive, etc) in the waist bag is still a somewhat cum- 
bersome prototype. Clearly it is experimental in nature, with 
the knowledge that trends in miniaturization, and new tech- 
nologies like conductive threads, conformal antennas, etc., 
will soon make this apparatus blend into ordinary clothing. 

The multimedia computer is connected to the internet over 
TCP/IP, using one or more forms of wireless communication. 
In particular, these include a 1987 WA4DSY system which 
provides a data rate of 56kbps, together with aG3RUH system 
with variable data rate, and an older dual band radio. The 
older system is essentially a backup system — higher output 
power together with a low data rate of 1200bps give it a 
robustness that keeps the author online, for example, in the 
sub-basements of buildings where the other systems do not 
work that well. 

Antennas are currently mounted in a hat which is lined 
with copper mesh, (the 1980 rig made use of a distributed 
antenna network sewn into a jacket). Work is underway to 
make the antennas completely unobtrusive, as they were in 
the 1980 rig, but while maintaining the same high level of 
performance of the current rig, through the use of conformal 
antenna technology. 

THE WEARCAM PERSPECTIVE ' WearCam' is an Internet- 
connected multimedia computer with its own Internet ad- 
dress, but the items are attached in a natural fashion, so that 
they can be used, at times, without (much) conscious effort, 
and while performing other activities such as walking, shop- 
ping, or riding a bicycle. 

The image from the camera(s) is presented on the display in 
a way that is natural and intuitive. A camera and microphone 
are attached to the display in such a way as to give a first 
person vantage — the camera "sees" exactly what the wearer 
sees, rather than the second person perspective of traditional 
multimedia applications where the camera "sees" a picture of 
the user. 

OTHER SENSORS The apparatus includes a ProComp 8 
channel analog to digital converter which facilitates mea- 
surement of voltages and such. In particular, the author's 
'smart shoes' (shoes with an array of sensors that provide in- 
formation about footstep force and velocity, etc.) and 'smart 
undergarments' that sense, for example, heart rate, respira- 
tion, and skin resistance, are connected to this analog to digital 
converter, providing the capability to log the information into 
a file, or to access it online. 

Other sensors such as infared and radar, enhance and ex- 
tend the author's sensory capabilities, and have been used for 
various experiments in synthetic synesthesia, which might 
someday be of assistance to the visually challenged[27]. 

The author's apparatus is somewhat reminiscent of the "Win- 
nebiko/Behemoth" work of Steve Roberts, N4RVE[28], ex- 
cept that it is built into clothing rather than a bicycle. 

'PERSONAL IMAGING* 'Personal imaging' is an important 
aspect of 'smart clothing', thus the camera and display are 
important parts of the apparatus. The current computer screen 
is actually quite similar to the display in the author's 1980 
version (it is still a miniature CRT operating at the same 5 to 6 
kV anode voltage), except that it provides high-quality color 



imagery; the field rate is three times as high (180Hz instead 
of 60Hz), and a color filter is used to sequence through red, 
green, and blue. This provides full color fidelity with good 
shadow detail as well as good highlight detail (LCDs often 
provide only washed-out pastel colors). A spinning color 
filter wheel was replaced with an electronic color shutter to 
reduce physical size and increase reliability by eliminating 
moving parts. 

EXPERIMENTS 

The apparatus, a wearable multimedia computer system equipped 
camera(s), and wireless Internet connection [29], enabled ex- 
perimentation with imaging applications in ordinary day-to- 
day situations, not just in a lab. Possible applications of this 
'WearCam* to the handicapped have been suggested [27], in 
particular, its use as a personal visual assistant (PVA), and as 
a visual memory prosthetic has been suggested. 

WearCam was originally designed and built as a "visual 
memory prosthetic' to attain an improved awareness of vi- 
sual experiences (e.g. to gain an enhanced awareness of light 
and shade), and to help overcome visual amnesia. Note that 
this goal is the opposite of that embodied in both virtual real- 
ity and personal sound systems (in the sense that they reduce 
awareness of actual reality). 

WearCam embodies a temporal visual filter that provides 
computer-induced flashbacks (possibly together with annota- 
tion). This functionality has made it possible to experiment 
with practical applications such as providing assistance in 
such visual tasks as remembering faces. 

EXPERIMENTS IN NETWORKED ONLINE COMMUNITIES 
Internet connectivity is due to antennas the author has erected 
on rooftops of various buildings. In particular, one of these 
is coordinated through the New England Spectrum Manage- 
ment Council to operate as an "open" gateway (available for 
use by the general ham radio community), and is listed as 
being an open gateway. 

Furthermore, with the current (or soon-to-be) availability of 
commercial systems such as Metricom, Wavelan, and Mo- 
torola, getting more 'smart clothing' online will soon be triv- 
ial. 

Experiments with two WearCam users have allowed an ex- 
change of viewpoint, so that each person sees through the 
other person's eyes[30]. 

Instead of just two people, suppose we have a community 
(network) of individuals wearing the apparatus. This could be 
a homogeneous community (all wearing the same form of the 
apparatus), but for simplicity of implementation, consider a 
heterogeneous community (Fig 5) wearing various prototypes 
of the apparatus. These prototypes are widely different in 
their design but provide similar functionality with ability to 
communicate with one another. 

EXPERIMENTS WITH THE VISUAL MEMORY PROSTHETIC 
WearCam presents the visual world to the wearer on a com- 
puter screen, so that the visual experience can be augmented, 
diminished, or otherwise altered, under program control if 
desired. This facilitates experiments on visual perception in 
which the experience of the visual world must pass through 
a 'visual filter* [30]. The 'visual memory prosthetic' is based 




Figure 5: The 4 safety net*: a network of individuals, three with WearCams: 
author at left with 1990 CRT-based WearCam; fourth and fifth (cellular 
hat) from left wearing LCD-based WearCams more recently designed and 
built by the author but lacking the dynamic range and color fidelity of 
the older CRT-based WearCam. Even this newer display technology is 
somewhat obtrusive. The displays are the most obtrusive parts of these 
systems (cameras, microphones, computers, and input devices of the three 
wearcams are almost completely invisible). It will take some time before a 
high-quality unobtrusive display becomes available. 

primarily on a 'visual filter 1 that doesn't (assuming proper cal- 
ibration) bend any rays of light passing through it, but only 
delays these light rays. Such a temporal-only 'visual filter' 
might, for example, function like a pair of eyeglasses made of 
hypothetical slowglass [3 1] and merely delay the lightfield. It 
might also produce a stroboscope or freeze-frame effect, as 
well as more sophisticated temporal visual filters such as for 
the face-recognition/reminder experiments to be described 
later. 

RESULTS AND DISCUSSION 

EDGERTONIAN EYES Early experiments with a variety 
of different 'visual filters' , involved experimenting with the 
WearCam apparatus in day-to-day activities[30]. Each of 
these filters provided a different visual reality. For one such 
filter, the author experimented by applying a repeating freeze- 
frame effect to WearCam. With this video sample and hold, It 
was found that nearly periodic patterns would appear to freeze 
at certain speeds. For example, while looking out the window 
of a car, periodic railings that were a complete blur without 
the apparatus were found to appear in sharp focus with the 
apparatus, creating a heightened sense of awareness of subtle 
differences between each period of a periodic structure — 
an awareness that exceeded even that attained standing still 
examining the structure. 

Rotational periodicity (such as in the blades of a spinning 
airplane propeller) would appear as objects rotating slowly 
backwards or forwards, in much the same way as objects do 
under the stroboscopic lights of Harold Edgerton[32]. 

FLASHBACKS AND FREEZE-FRAMES Of greater interest 
than just being able to see that which is invisible to the naked 
eye, was the fact that sometimes the effect would cause certain 
things to be rememembered much better. In particular, it was 
found that faces would be remembered much better with this 
freeze-frame effect 

DEJA VU A free-running visual memory prosthetic was 
found to be helpful in way-finding. With the ability to com- 
pare past and present imagery, it became quite evident when 
one had been in a particular place before (e.g. if one were 
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going around in "circles"). When this comparison is done 
automatically, the computer attempts to compare all previ- 
ous images with the current image (from what the camera 
is currently pointed at). With some form of visual process- 
ing, WearCam can also function as the visual equivalent of 
Starner's rememberance agent[33] (a text interface that con- 
stantly watches what the user types and automatically reminds 
the user of related text files). 

VISUAL CLEW* The * visual clew 2, is a computer-assisted 
way-finding system that runs on WearCam. 

We've all no doubt been lost at one time or another. One 
might enter a shopping complex, or the like, and be unable to 
find the way back to the car or subway stop at the end of the 
day. 

It was found that by building up a stack of images, ei- 
ther consciuosly deciding to capture an image at each branch 
point, or putting the apparatus in a free-running mode, the 
problem of getting lost was solved, at least to the extent that 
one could find one's way back to where one started from, 
whenever desired. 

The current implementation of 'visual clew* uses the Wear- 
able Wireless Webcam: each image is appended to the web 
page, and then, when it is time to return (e.g. to find my way 
back to the parking lot) the Web browser (Mosaic) is invoked, 
to browse through pages of previously captured images. Al- 
ternate implementations have been built using a local image 
stack but the Web stack allowed for shared "visual memory", 
so that, for example, two people could use such an apparatus 
to find each other, should their paths happen to have crossed 
at some point. 

A method of determining if images are in the same orbit of 
the projective group of coordinate transformations has been 
described[34][20]. This 'video orbits* approach, together 
with other image recognition systems, can be used to au- 
tomate the way-finding 'visual clew* system. In particular, 
if one applies the following use strategy, it is very easy for 
even a small battery-powered computer with only moderate 
processing capability to recognize a previous image: 

1 . Walk down the center of every corridor whenever possible. 

2. Capture an image at each branch point, and, if desired, capture 
additional images along the corridor whenever it is possible to stand 
in the center of the corridor (e.g. if there are obstructions, wait until 
it becomes possible to be recentered). 

3. When possible, capture images that have the fewest number of peo- 
ple in the scene. 

Thus when the same path is encountered again, if the frame 
rate (rate at which images can be captured and compared to 
previous orbits) is sufficient, images will lie in the same orbit 
of the projective group of coordinate transformations, and be 
automatically identified as such. 

Alternatively, instead of capturing a single image at each 
branch point, a partial environment map may be captured at 

2 The M clew" metaphor is derived from Greek mythology, in which The- 
seus used a clew (ball of thread) to retrace his path back out of the "labyrinth 
of perversion". 



each branch point (stop for a couple of seconds, and 'paint 
with looks' — making a quick glance around to generate an 
image composite). Then if the same location is encountered, 
the two collections of input images will be in the same orbit 
of the projective group of coordinate transformations. 

ANNOTATED 'FLASHBACKS': WEARABLE FACE-RECOGNIZER 
Many people (author included) have difficulty remembering 
faces. The 'visual memory prosthetic' was found to assist in 
the task of remembering faces, through the use of computer- 
induced flashbacks. In addition to helping the human remem- 
ber, and therefore recognize, faces, computers are also capa- 
ble of directly recognizing faces. Previous work by others 
is based on using a fixed, tethered camera [35] [36] [37] [38]. 
The kinds of applications for this work might include video 
surveillance with a fixed camera and people moving through 
its field of view. The FBI-funded FERET project comprises 
a large database (more than 7000 faces) that can be searched 
quickly on a workstation-class system. 

Automatic face recognition has raised extensive privacy 
concerns[39]: "Privacy International is calling on the UK govern- 
ment to prohibit . . . Computerised Face Recognition (CFR) systems 
that have the capacity to automatically compare faces captured on 
CCTV, with a database of facial images. Several police and com- 
mercial organisations are developing this technology. . . . should 
. . . re-establish some democratic mechanism in the development of 
wide-scale urban CCTV systems . . . grave risk that the CCTV in- 
dustry is out of control. . . quashing public debate. . . the systems 
have challenged some fundamental tenets of justice, and created the 
threat of a surveillance society. Other more traditional approaches 
to law enforcement and social justice are being undermined. . . " 

However, computational resources, attached to a person, 
suggest the possibility of turning the tables on the traditional 
third-person perspective (such as a ceiling-mounted surveil- 
lance camera), and, instead, using face recognition from a 
person-worn perspective. In particular, if face recognition is 
used ubiquitously by ordinary individuals as part of their day- 
to-day living, it will give rise to a more democratic society 
in which policemen, as well as ordinary citizens (and shop- 
keepers as well as shoppers, bank tellers as well as bankers) 
will be recognizable and accountable for their actions. 

A variety of face-recognizer implementations are possible 
with the current WearCam apparatus. These range from stor- 
ing the database of candidate faces on the body-worn ap- 
paratus and using local processing, to connecting remotely 
from the apparatus to the database, possibly using remote 
processing. 

Several different implementations of the capture/display 
configuration (e.g. having both the camera and display ro- 
tated 90 degrees, having the camera rotated 90 degrees with 
the display still in landscape orientation, etc) were tried. It 
was found that the best overall configuration was to have 
the camera rotated 90 degrees (portrait) but with the display 
still in landscape orientation. Anecdotes on the author's ex- 
periences "living in a rot90 world" (reminiscent of George 
Stratton's upside-down glasses [40]) appear in [30]. 

Improvements to the 'wearable face recognizer , included 
providing means of alignment, using a registration template 
(Fig 6) displayed on top of the video to facilitate manual 
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Figure 6: Template-based wearable face- recognizer (a) As candidate approaches, an effort is made to orient the apparatus (by turning of the head) so that the 
candidate is centered. This is easy because the full-motion color video input stream appears on the computer screen together with the template, (b) At some 
point, the distance to the candidate will be such that the scale (sire of the face on the image plane) will be appropriate, and, while still keeping the orientation 
appropriate, the match is made, (c) After the match is made, the template image drops away, revealing a radioteletype (RTTY) window behind it, upon which 
is displayed the desired information (for example, the name of the candidate, "Alan Alda'\ and possibly additional parameters or other relevant information). 

alignment of the face, through rotating the head to aim the 
camera appropriately, (The cursors themselves were imple- 
mented as a JPEG image.) Further implementational details 
of the wearable face recognizer and visual memory prosthetic 
are available in[27] and [30]. 

VIDEO ORBITSfPERSONAL IMAGING* When shopping, 
friends and relatives can remotely look at whatever one is 
looking at (the author has been surprised from time to time, 
for example, with unexpected email about the fruits and veg- 
etables in the WearCam's view, or a reminder to pick up some 
milk, which was especially surprising when, for example, a 
remote visitor happened upon a portion of the environment 
map that was captured by accident). 

In more recent years, with the advent of the World Wide 
Web, a new form of connectivity has been explored using the 
'Wearable Wireless Webcam' (http://wearcam.org). 
The application of WearCam has been an exploration of a new 
form of personal visual connectivity. As an tool for visual 
artists (in some sense the pencigraphic image composites are 
an expressive art form), such a device can reduce the time 
from first seeing something of visual interest, to showing an 
image in a gallery (completed exhibition), down to a fraction 
of a second. 

ILLUSORY RIGID PLANAR PATCH Examples of illusory 
rigid planar patches arise directly from projective coordinate 
transformations applied to individual images. In particular, 
the wearcam apparatus displays both video (imagery) and text 
on the NTSC-svideo screen. Typically a large font (e.g. such 
that there are on the order of 30 characters across) is used so 
that the text will be very easy to read even when one is jogging 
or the eyeglasses are being jostled around. An example of a 
text window superimposed on an image composite appears 
in Fig 7. 

Face recognition may be used together with 'pencigraphic 
imaging*. In particular, the group of projective coordinate 
transformations may be used to sustain the illusion of a rigid 
planar patch. Thus once the face is recognized by the com- 
puter, an virtual name tag is created using the illusory rigid 
planar patch. Then when the wearer looks to the right, for 
example, causing the video imagery to move to the left across 
the image sensor (and viewer's screen), the text moves to the 




Figure 7: Here the author is conducting banking business, and a problem 
has been encountered with the author's bank card. A radioteletype (RTTY) 
session appears as a rectangular text window on the sixth image in a seven- 
image pencigraphic composite, creating the illusion of a rigid planar patch. 
Text is sideways because the author had adapted to seeing images rotated 90 
degrees — text appeared normal (unrotated) to author The second window 
on the right denotes the viewpoint of another person, remotely (the author's 
spouse). This viewpoint, corresponds to the lower right hand quadrant of 
the first image in the sequence. Although the author's gaze is currently 
fixed on the paperwork in front of him, the remote viewer is navigating the 
environment map independent of where the author is looking. In particular, 
the remote viewer recognizes the face of the teller serving the customer to the 
author's right, and sends a message into the author's RTTY window, upon 
his current point of gaze. Note also the 3 video surveillance cameras visible 
at the top of the image composite. This would suggest that both the author 
and the bank have a visual record of the transaction. In the future, just as 
both parties of such a transaction keep signed copies of all documents, so 
too will both parties keep 'visual memories' of the transaction, giving rise 
to balance between wearable technology and environmental technology. 
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Figure 8: Six frames from an image sequence in which author looked from 
left to right. The illusory rigid planar patch is created by featureless tracking 
of the video orbit of the projective group of coordinate transformations. 
Notice that in frame six, even though the face is no longer in view of the 
apparatus, it continues to be 'tracked*. What is actually being tracked is 
the entire orbit (the projective coordinate transformation arising from other 
objects in the room). Here the illusory rigid planar patch is a list of grocery 
items previously purchased by the author from this same cashier. Such an 
apparatus may be useful when returning items to a department store, as it 
assists one's memory of what was purchased, when, and from whom. A 
clear memory of details often helps in avoiding disputes regarding terms of 
the sale. 

left to follow it. But not only does the text move to the left, 
but it also "keystones" and 'rechirps' itself to maintain the 
illusion of a ridid planar patch in 3-D space (Fig 8). 

ACTIVE BADGES AND PERSON-TRACKING Card keys 
and active badges both represent technologies that either keep 
track of where individual people are located, or comprise 
hardware that has the potential to do so. 

MIT has attempted, somewhat unsuccessfully, on two differ- 
ent occasions, to deploy a card-reader system for access con- 
trol to various buildings. In both cases, students have strongly 
opposed the initiative. In particular, there was widespread 
concern regarding the privacy implications of such a system. 

TURNING THE SYSTEM INSIDE-OUT Both the card key 
and active badge systems rely on a "smart" element built into 
the architecture (card reader or IR receiver) and a "dumb" 
element (card or beacon) carried or worn by the user. The 
"smart" element is networked to a central computer system, 
while the "dumb" element has no communications or net- 
working capability whatsoever. 

Suppose, however, that we swap the two. Suppose that the 
user carries or wears the "smart" element, and the building 
architecture is endowed with the "dumb" element. Thus, for 
example, the user might wear the infa red (IR) receiver, and 
have this connected to his/her 'smart clothing', while numer- 
ous beacons would be distributed throughout the building. 
This means that there is no need to network the beacons, no 
need to wire the building. The system relies on the commu- 
nications infastructure each user wears. 

However, now the location of the user is known to the user's 
clothing, and thus the user has control over who can and can- 
not know his/her location. A user might, for example, define 
an access control list comprising faculty advisor, thesis advi- 
sor, colleagues, etc.. The user's clothing would automatically 
encrypt the user's location (as determined by the last beacon 
"seen" by the user's clothing) and transmit this information 
to the desired recipients. Any interception of this commu- 
nications would be unintelligible to those not on the access 
control list. 

In a prototype system, the author deployed a number of 
"room tags" — name tags fixed at known locations in the 
building (Fig 9(a)), and fixed an .IR receiver to his glasses 
(Fig 9(b)), and connected this to the clothing-based computer. 



In addition to giving the user control over his/her personal 
whereabouts, the system may also be used to provide location- 
dependent computer-induced flashbacks (Fig 9), adding new 
dimensions to the visual memory prosthetic. For example, 
entering my office, I was surprised to find that the lost sweater 
I had been looking for that day had been sitting on my desk 
only a short while ago. Thus I 'remembered' that I had worn 
it that day, and that therefore I must have lost it somewhere 
nearby. 

ACHIEVING BALANCE: COOPERATION BETWEEN SMART 
SPACES AND SMART CLOTHING The proliferation of ever 
more intrusive environmental technology has created an im- 
balance between individuals and their environment. This 
section presents some examples of how wearable technology 
might work cooperatively with the environmental technol- 
ogy. An example of environmental control (HVAC) using 
smart underwear is presented. 

SMART UNDERWEAR AND BIOSENSORS Even when we 
take off our 'smart clothing' at night, we might still choose 
to keep on our 'smart underwear* which controls the heater 
or air conditioner in the room. Upon arriving home, late at 
night, one is generally too hot from just climbing the stairs, 
etc., so when first going to sleep, the underwear tells the 
heater to turn off, but after a couple of hours sleeping, when 
one's metabolism slows down, the underwear senses the re- 
sulting changes in one's body temperature/conductivity, and 
turns up the heat. Our clothing of the future may some day be 
interoperable and interconnected, so that it keeps track of our 
physical condition and allows us to decrypt this information 
for evaluation by a doctor or other professional of our choos- 
ing. Further description of the 'smart underwear' prototype, 
and anecdotes on the author's experience designing and using 
it is appears in[41]. 

The natural place for our medical records is right in our 
clothing. Having a patient wear his or her entire medical 
history would solve much of the medical records privacy 
problems we face today. With various biosensors, the most 
current and up-to-date information would be readily available 
withing the very clothing that's taking the measurements 3 . 
This approach would eliminate the need for, and the possi- 
ble abuses that can arise with, a central database of medical 
records, and would eliminate the need to for a person to 
venture through bureaucratic procedures to access his or her 
own medical information. It would also eliminate the prob- 
lems associated with smart cards, as clothing is almost always 
worn, while cards may be misplaced and inaccessible in times 
of emergency care. Epidemilogical research would still be 
possible being done with the patient's data — participating 
patients could make the data accessible to organizations do- 
ing the research, but this would be done through a query to 
each participating patient's online 'smart clothing* each time 
the data were needed, so that the patient's clothing would be 
kept "in the loop", that is, access logs would be automatically 
generated in the 'smart clothing', so that patients could trace 
the history/usage of their data at a later date if desired. 



3 Of course, one would want to have one's medical records replicated 
(backed up) in the clothing of selected friends and relatives, to prevent data 
loss in the event of clothing failure. 
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Figure 9: A paradigm reversal for person- tracking (a) 'Room tag* (one of many name tags that may be easily deployed throughout the building), (b) 
Paradigm reversal: tags and receivers swap places. Receiver is attached to author's eyeglasses and plugged into 'smart clothing*, (c) Computer-induced 
flashback triggered by revisiting same room as before. Note name tag is visible in upper left of picture. Induced 'memory' of sweater, sitting on desk in lower 
left corner of picture, proved to be helpful. 



CRIMEWATCH AND 'SAFETY NET In a networked wear- 
able multimedia community, people might pay attention mostly 
to their immediate surroundings, but may, at times, get an im- 
age from someone who thinks there might be danger. This 
fear of danger might be triggered by a 'maybe I'm in distress' 
button pressed by the wearer, or automatically. Automated 
distress may be facilitated through a heart rate monitor and an 
activity meter/pedometer, such as the sensors in the author's 
shoes. The heart rate divided by the footstep activity gives a 
'visual saliency' index. Should someone pull out a gun and 
demand cash from the wearer, the 'smart clothing* might re- 
spond appropriately (video capture/transmission at maximal 
frame rate, etc.) by virtue of the sudden increase in heart rate 
for no apparent reason (e.g. without any increase in phys- 
ical exertion). As a personal safety device, ubiquitous use 
of 'smart clothing 1 might have the potential to turn the world 
into a small-town community — a global village as barriers of 
time and space fall. A community of individuals networked 
in this way would look out for each others' safety in the form 
of a 'neighbourhood watch*. This 'safety net' could be used 
for a 'virtual safewalk': a participant, about to walk home 
or enter an underground parking garage late at night, sees 
'eye-to-eye' with one or more people (perhaps in a different 
time zone, say somewhere in the world where it is morning, 
so the virtual escort has fresh alert eyes). Neil Stephenson's 
Global Neighborhood Watch[42] comes to mind, but now 
with free-roaming tetherless connectivity. 

This level of connectivity raises many new social and eth- 
ical questions that will need to be addressed as we further 
experiment with more WearCams. 

SMART CLOTHING v. SMART UNIFORMS 'Smart cloth- 
ing' represents a significant future direction for computing. 
The recent proliferation of wearable computers (there are 
about 5 companies making wearable computers now) sug- 
gests that we're moving in that direction. However, many 
of the applications of wearable computers so-far envisioned, 
such as the land warrior (military), the intelligent mainte- 
nance aid, or various applications in the workplace[l 5] might 
better be described as 'smart uniforms'. A 'smart uniform' is 
issued to a soldier or employee at the start of a job, and then 



taken away after the job is completed. 

There is a fundamental difference in the way that people 
feel about their own clothing as compared to a uniform. Al- 
though people can become quite familiar with their uniforms, 
whether worn in prison, the military, certain workplaces, or 
old-fashioned schools, the individuality of personal clothing, 
and the pleasures associated with its selection and wearing 
should be extended to computing. The full power and en- 
joyment of this synergy between human and machine will 
be realized only when the computer is owned, operated, and 
controlled by the wearer, giving rise to truly personal com- 
puting. Indeed, examples of wearable technology at the ex- 
treme opposite to the personal wearable, are the wearable ID 
transponders that have been rejected by many employees, and 
the devices attached to criminals to keep track of them[16]. 
These devices are owned, operated, and controlled by a re- 
mote entity. Some such devices even have the capability to 
provide the wearer with an "electrical corrective signal" (eu- 
phemism for electric shock) when the wearer does something 
against the will of the entity that controls the system (e.g. 
ventures outside a prescribed boundary). This prospect is * 
at least as troublesome as the pole-top surveillance cameras 
discussed earlier. 

ANECDOTAL DISCUSSIONS 

ON THE "SAFETY VERSUS PRIVACY" ARGUMENT When 
I first joined the MIT Media Lab, I expressed concern regard- 
ing the possible development of surveillance technologies, 
such as ubiquitous use of video cameras, face recognition 
and the like. My advisor, trying to relieve my concerns re- 
garding a possible Big-Brother future, presented me with the 
argument of her advisor (Sandy Pentland) who was the di- 
rector of the research on face recognition: "Cameras make the 
world a smaller place, kind of like a small town. You give up privacy 
in exchange for safety. In a small town, if you were suffering from 
a heart attack and collapsed on the floor of your kitchen, chances 
arc better that someone would come to your rescue. Perhaps a 
neighbour would come over to borrow some sugar, and, since your 
door would be unlocked, would just come right in and see you had 
collapsed and come to your aid." Although this analogy makes 
perfect logical sense — on the safety versus privacy axis, the 
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small town of the past and future with ubiquitous surveillance 
are very similar — there was still something very disturbing 
about it. 

If we look along a different dimension, characterized by 
symmetry, the small town and a future with ubiquitous surveil- 
lance are exact opposites. In a small town, the sheriff knows 
what everyone's up to, but everyone also knows what the 
sheriff is up to. This symmetry is generally not the case with 
regard to surveillance systems like the ones used in the UK, 
which, although often installed by governments, are operated 
as closed (secret) systems, the imagery being unavailable to 
ordinary citizens. 

Phil Patton [5] discusses the surveillance dilemma, mak- 
ing reference to the ubiquitous "ceiling domes of wine-dark 
opacity", making mention that "many department stores use 
hidden cameras behind one-way mirrors in fitting rooms", 
and in general, that there is much more video surveillance 
than we might at first think. 

With so much video surveillance in place, and growing at 
a tremendous rate, one wonders if privacy is a lost cause. If 
we are going to be under video surveillance, we may as well 
keep our own "memory" of the events around us, analogous 
to a contract in which both parties keep a signed copy. Falsi- 
fication of video surveillance recordings is a point addressed 
in the movie Rising Sun, and in William Mitchell's book, 
The Reconfigured Eye [43]. However, if there is a chance 
that individuals might have their own account of what hap- 
pened, organizations using surveillance would be much less 
likely to risk falsifying surveillance data. Even though it is 
easy to falsify images [43], when accounts of what happened 
differ, further investigation would be called for. Careful anal- 
ysis (e.g. kinematic constraints on moving objects in the 
scene, the way shadows reflect in shiny surfaces, etc) of two 
or more differing accounts of what happened would likely 
uncover falsification that would otherwise remain unnoticed. 
The same technology that is used to demonstrate a person 
has removed an item from a department store without paying 
may be used by a person to demonstrate that he or she did, in 
fact, pay. One can only imagine what would have happened 
if the only video recording of the Rodney King beating were 
one that had been made by police, using a network of po- 
lice surveillance cameras, such as the camera networks used 
in some cities in the UK[44]. Of course, most officials are 
honest, and would have no reason to be any more paranoid 
of an average citizen's camera operating together with their 
own network of cameras. 

Once we accept that images can be easily falsified, we will 
someday soon (some have already) begin regarding images 
as having no more truth than verbal or written accounts. The 
difference between whether images themselves are presented 
to a jury from someone's WearCam, or a verbal description 
of incredible detail is provided by that same person through 
their looking into their own WearCam's memory, will matter 
less and less as technology evolves. 

Privacy concerns regarding WearCam itself have also been 
raised. However, living in a society in which cameras were 
not permitted, unless they were worn on people, would be 
far preferable to today's society — then at least we'd know 



we had privacy when we were alone — our instincts like 
"don't pick your nose in front of other people" would work 
in harmony with ubiquitous personal imaging. 

CONCLUSIONS 

'Smart clothing', a wearable multimedia computer and 'per- 
sonal imaging* system been proposed. Its practical utility 
in day-to-day applications has been explored, in particular, 
through experiments in way-finding, face recognition, cre- 
ation of partial environment maps and shared visual spaces, 
etc. A framework for 'personal imaging' has been set forth, 
in particular, through the use of a wearable camera system, re- 
garding the camera as an array of directional lightmeters, and 
through consideration of 'video orbits' (orbits of the projec- 
tive group of coordinate transformations acting on images). 

Much like the cellular telephone, pager, pocket calculator, 
notebook computer, pocket organizer, wristwatch, etc., that it 
will subsume and replace, the most successful wearable tech- 
nology will be owned, operated, and controlled by the wearer. 
Much like the the wearer's own clothing, this technology will 
arise out of the wearer's own choosing. 

'Smart clothing' raises some interesting ethical and social 
issues, which we will need to face when the apparatus makes 
its way into widespread use. The boundaries between seeing 
and viewing, and between remembering and recording will 
crumble. When we purchase a new appliance, we may well 
'memorize* the face behind the store counter. A week later, 
our spouse, taking the appliance back for a refund, could 
'recall' the name and face of the clerk she never met. 

In some sense 'smart clothing' turns the tables on video 
surveillance — the shopkeeper who illegally chains fire ex- 
its shut will risk getting caught on candid camera just as the 
shoplifter risks getting caught on the shop's surveillance cam- 
eras. Although miniature camcorders have brought to light 
incidents such as the Rodney King beating, they lack wire- 
less communications to safeguard the image data from being 
siezed or destroyed. 

Because WearCam is tetherless, it has been possible to wear 
the apparatus for several years, in day-to-day interactions. 
Getting multimedia computing off the desk and onto the 
streets, shops, and into other establishments, has raised some 
interesting social issues. 

'Smart Clothing' offers an alternative to centralized surveil- 
lance. It suggests a future in which people, through prosthe- 
sis, might have both improved visual memory and improved 
ability to share it. But it also suggests a hope that the visual 
memory be distributed among people, and be less likely to 
be abused than if it exist in a centralized form, as is more 
common with a network of surveillance cameras, such as is 
commonly used on the streets in the UK. The proliferation 
of hidden cameras everywhere has the possibility to threaten 
our privacy, but suppose the only cameras were the prosthetic 
elements of other individuals. Then at least one would still 
have privacy when one was alone. 

Furthermore, as images become easier and easier to edit [43], 
WearCam, like other cameras, will begin to be regarded as 
being more like a visual memory aid and an artist's tool than 
a source of evidence of fact. 
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